The present invention relates to an image-feature extraction method of extracting feature parameters representing features of an image from the image. More particularly, this invention relates to an image-feature extraction method of enabling extraction of image-feature parameters suited to human sensibility on similarity from the image and also of automating a work to extract feature parameters from the image when feature parameters representing features of an image are to be extracted from the image.
Recently, in association with development of computer technology and image processing technology, accumulating enormous amount of electronic images to construct an image database have actively been attempted. When constructing such an image database, the most important thing is the availability of an image retrieval method so that a desired image can easily be acquired from a large number of accumulated images.
As one of the image retrieval methods as described above, there is one comprising the steps of allocating a search key with some term or terms to each image, verifying a keyword inputted in the same method as that for document retrieval with the search key allocated to the image, and returning the image with the search key matching the keyword as a search result.
However, an impression which a person from about an image varies from person to person. Therefore, a search key allocated to an image does not always cover every impression a person has from the image. Thus, so that there are many cases in which a satisfactory image can not be acquired in image retrieval using a search key based on terms inputted by an operator. The reason behind this is that, people can not find in many cases appropriate terms to express features concerning a shape of things such as a person, animal, or a building appearing in the image (which are collectively called as xe2x80x9cobjectxe2x80x9d hereafter) and a positional relation among the objects.
Therefore, an image should not be retrieved based on a result of verification of a keyword against a search key with terms, but it is desirable to construct a system so as to provide a particular image (described xe2x80x9cquery imagexe2x80x9d) as a criteria for searching and enable retrieval of an image which is similar to the query image from the image database. To realize the retrieval method described above, there has been proposed a method in which feature parameters representing features of each image are retrieved from the query image as well as from an image as a target for retrieval, similarity between the feature parameters of the query image and those of the image as a target for retrieval are determined, and the image having the feature parameters similar to those of the query image is returned as a search result. Namely, this image retrieval method is a method of retrieving a similar image based on the idea that the similarity between features is equivalent to similarity between images themselves.
There are two papers that disclose such a retrieval method. The first paper discloses a query method of analyzing images themselves, extracting colors (a color histogram), texture, and shapes of objects or so in the images as feature parameters of each image to construct a database, and acquiring an image with specified feature parameters by specifying the feature parameters of the image in the database when the image is queried. This paper is by Myron Flickner et. al., xe2x80x9cQuery by Image and Video Content: The QBIC Systemxe2x80x9d Computer, September 1995, pp 23-32.
The second paper discloses a method of retrieving a similar image by generating a color histogram of each image using a cylindrically-segmented HSV color space and by using the generated color histogram as feature parameters of each image. This paper is by John R. Smith and Shih-Fu Chang, xe2x80x9cTools and Techniques for Color Image Retrievalxe2x80x9d SPIE, Vol. 2670, pp426-437.
With the image query method disclosed in the first paper, it is possible to query an image by using feature parameters of various images. However, when feature parameters are extracted from images and similarity between the images is determined based on the extracted feature parameters, the feature parameters do not fully cover the scale in which a human determines the similarity. Therefore, precision of retrieval is rather low. For example, feature parameters extracted based on the shape of an object only expresses how much the object resembles a circle (circularity) and in which direction the object is compressed (central moment). Namely, the extracted features are so rough that it is impossible to perform image query based on appropriate and detail impressions which a human has when looking at an object in the image. Further, there is a great possibility that the retrieval result contains noise.
Further, in the first paper, it is possible to perform image search based on a shape of an object as feature parameters of the image. However, a user is required to specify a locus of an edge of the object existing in the image when the object is extracted from the image. This requires an extra effort. An object in the image is one of the most noticeable feature when a person looks at the image. Therefore, by extracting a shape of an object as image-feature parameters and using the parameters for image retrieval, it is possible to perform high precision image retrieval based on the impressions which a human has from the image. However, as extra efforts are required, an enormous amount of manpower and time are required to construct an image database with a large amount of images accumulated therein.
In the second paper, the HSV color space used for extracting a color histogram from an image as feature parameters of the image is simply segmented into cylindrical shapes, so that some wastefully segmented areas are generated and increase in the amount of data is disadvantageously caused. More specifically, when there are a small number of segments in the color space, even colors dissimilar to each other may sometimes be determined as similar ones, and precision in determination of similarity between images gets worse. On the other hand, when a number of segments in the color space is too large, the amount of data for the color histogram becomes too much, and a long time is required for computing the similarity between the images. In addition, when the number of segments in the color space is too large, the colors which a human feels the same as or similar to each other may be determined as dissimilar ones. The method disclosed in the second paper corresponds to the case where the number of segments in the color space is too large, and the method includes the problems that a long time is required for computing similarity, and that human sensibility on feel are not sufficiently satisfied.
Namely, it may be said that, in the conventional technology, impressions which a human acquires from an image are not sufficiently reflected to feature parameters extracted from the image and an operational environment suited for use is not provided. For instance, a heavy work load is required for extracting feature parameters from an image. Further it may be said that, in the conventional technology, impressions which a human acquires from an image are not sufficiently reflected to features which are extracted from the image, and that it is difficult to acquire a result of retrieval as desired by an operator. In other words, in the conventional technology, attention is not focused on a human as an operator, and a user-friendly environment for working is not provided. To improve convenience in working, feature parameters should be extracted from an image without requiring manpower, and the extracted feature parameters should be suited to human sensibility on similarity, and whether a retrieved image completely or substantially satisfies the conditions for searching set by a human should be determined based on human sensibility on similarity.
It is an object of the present invention to automate, when feature parameters representing features of an image are extracted from an image, the work of extracting feature parameters therefrom, and to enable extraction of feature parameters suited to human sensibility on similarity without requiring any specific work load to an operator.
It is another object of the present invention to improve precision in image retrieval by making it possible to extract image-feature parameters suited to human sensibility on similarity for acquiring a result of image retrieval suited to human sensibility on similarity.
It is still another object of the present invention to speed up image retrieval by segmenting a color space into a plurality of areas suited to human sensibility on color, when a color histogram is extracted from the image as image-feature parameters, for suppressing a number of segmented color spaces to the minimum level, and also by using a color histogram.
An image parameter extraction method according to one aspect of the present invention comprises the steps of identifying to which area in a color space with a plurality of areas previously segmented thereinto each pixel belongs based on a color of each pixel which forms a prepared image; and generating a color histogram of the image based on the number of pixels which belong to each of the areas. When the generated color histogram is extracted as feature parameters representing features of the image, a color space which is suited to human sensibility on color and also which is segmented into areas each suited to human sensibility on color is used as a color space. Since the color space suited to human sensibility on color is used from the initial step, image-feature parameters suited to human sensibility on similarity can be extracted without requiring any heavy work load to an operator.
An image-feature extraction method according to another aspect of the present invention comprises the steps of identifying to which area in a color space with a plurality of areas previously segmented thereinto each pixel belongs based on a color of each pixel which forms a prepared image; generating a color histogram of the image based on the number of pixels which belong to each of the areas; when the generated color histogram is extracted as feature parameters representing features of the image then accumulating a number of pixels belonging to other areas in the number of pixels belonging to each of the areas according to color similarity between the areas; and generating a color histogram based on the number of pixels belonging to each of the areas after accumulation thereof. Since a correlation between bins can automatically be checked and identified by a program, image-feature parameters suited to human sensibility on similarity can be extracted without requiring any heavy work load to an operator.
An image feature extraction method according to still another aspect of the present invention comprise the steps of identifying to which area in color space with a plurality of areas previously segmented thereinto each pixel belongs based on a color of each pixel which forms a prepared image; generating a color histogram of the image based on the number of pixels belonging to each of the areas; when the generated color histogram is extracted as feature parameters representing features of the image then previously setting a typical color of each of the areas; computing similarity between a color of each pixel and a typical color of each area; and determining a number of pixels which belong to each of the areas according to the computed similarity. Since a correlation between bins in the above-described method can automatically be checked and identified by a program, image-feature parameters suited to human sensibility on similarity can be extracted without requiring any heavy work load to an operator.
The image feature extraction method according to still another aspect of the present invention comprises the steps of extracting an object from a prepared image; when feature parameters representing features of an object are extracted from the extracted object then setting points evenly spaced along an outline of the extracted object; computing displacement of an angle formed with lines linking between adjacent points; and extracting each computed displacement of the angles as feature parameters of the object. Since feature parameters of an object can automatically be extracted by a program in the method as described above, image-feature parameters satisfying human sensibility on similarity can be extracted without requiring any heavy work load to an operator.
The image-feature extraction method according to another aspect of the present invention comprises the steps of extracting an object from a prepared image; when feature parameters representing features of an object are extracted from the extracted object then extracting a color of each pixel which forms the image and clustering the colors in a color space; identifying any of the clusters obtained by the clustering including a color appearing most frequently in a previously specified area of the image as a cluster for a background area of the image; and extracting an area of the image formed with colors which belong to the clusters other than the identified cluster as an object. Therefore the work to extract image-feature parameters from the image can be automated.
A program for making a computer execute the steps of the image-feature extraction method as described above is recorded in a computer-readable record medium. Therefore, through execution of the program by a computer, image-feature parameters suited to human sensibility on colors can be extracted, and also a high precision result of retrieval satisfying human sensibility on similarity can be obtained by using the extracted feature parameters.